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Abstract 


Objective: to have obstetrical nurse experts analyze the composition of the instrument for nursing care technology in the 
prevention and management of hemorrhage in the third stage of labor and apply assessment criteria to validate it. 
Method: Methodological, quantitative, descriptive study. The instrument, based on an integrative and a systematic 
review, was divided into five axes, 20 domains, and 81 items. The instrument was sent to experts who assessed it using 
an electronic form (Google forms) with a seven-point Likert scale ranging from 7-totally agree to 1-totally disagree. 
Analysis included Cronbach’s Alpha coefficient index (> 0.70) and Content Validity Index (CVI> 0.80). 

Results: All ten criteria used to assess the instrument, which were analyzed by the 20 experts, presented very high 
Cronbach’s Alphas. The criteria were: clarity; coherence; scientific writing; relevance; sequence; homogeneity and 
current ness (Alpha 0.96); scope; criticality of items, and objectivity (0.95), that is, the experts considered the items to 
have very high reliability. The CVIs were also considered to be excellent. The items understanding, consistency, criticality 
of items, scientific writing, relevance and current ness of content obtained the highest CVIs (0.95); sequence and 
homogeneity obtained a CVI equal to 0.90; and objectivity and clarity presented the lowest index: 0.85. 

Conclusion: The instrument was validated by obstetrical nurses and considered to be highly reliable (total alpha equal to 


0.96 and CVI equal to 0.92). 
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Introduction 


Maternal mortality is an indicator of the life conditions 
and health care provided to a population; almost all such 
deaths are avoidable and occur in developing countries. 
Maternal mortality indicates gender inequalities, poor 
access to education, to proper nutrition and health, and its 
preventable nature should be taken into account when 
inter sectoral actions are implemented to ensure access to 
quality health care for the entire population [1]. The 
worldwide maternal mortality ratio (MMR) fell 
approximately 44% in the past 25 years, from 385 
deaths/100,000 live births in 1990 to 216 
deaths/100,000 live births in 2015. The goal for 2015 was 
not achieved in Brazil; there should have been a decrease 
to a maximum of 35 maternal deaths/100,000 live births 


[1]. 


Postpartum hemorrhage (PPH) is the most common 
form of obstetrical bleeding and one of the primary causes 
of maternal morbidity and mortality. Obstetrical 
hemorrhage is the main cause of maternal deaths 
worldwide. Its incidence varies among countries but, 
according to the World Health Organization, it accounts 
for 27% of the direct obstetrical maternal deaths, 
especially in the postpartum stage; in some countries, it 
accounts for more than 50%. Even in developed countries, 
hemorrhaging is among the three primary causes of 
maternal death [2-7]. Even though PPH rates vary 
according to the economic conditions of countries 
(between high- and low-income countries), PPH is 
estimated to affect 6% of births [8]. 


Even though the use of uterotonics is a preventive 
measure used to treat postpartum hemorrhage [9,10], 
there has been an increase in the global PPH rates in the 
last 25 years in countries such as Australia, Canada, 
United Kingdom, Ireland and the United States [11-13]. 


Primary hemorrhage is defined as excessive bleeding 
in the first 24 hours after birth, blood loss greater than 
500ml after vaginal delivery and blood loss greater than 
1,000ml after a C-section. PPH may be minor (500- 
1000ml) or major (more than 1,000ml). Major PPH is 
classified as moderate (1000-2000ml) or severe (more 
than 2000ml) bleeding. Secondary PPH is abnormal or 
excessive bleeding from the birth canal between 24 hours 
and 12 weeks after birth [5]. 


Due to its high impact on women’s and infants’ health, 
decreasing the rates of PPH is one of the main goals of 
international health organizations. One of the strategies is 
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to properly identify women who are at greater risk of 
experiencing excessive blood loss after birth, in an 
attempt to promote a more rigorous surveillance of this 
population. Even though obstetrical bleeding may occur 
unexpectedly, some studies report specific risk factors for 
PPH [13-19]. 


All pregnancies, even 20-week pregnancies, may 
present the risk of PPH. Most cases do not present 
identifiable risk factors, thus monitoring and early 
identification of bleeding is necessary. Antepartum risk 
factors account for 65% of PPH cases and include being 
older than 40 years of age; an only child; having a 
suspicion of or having premature placental abruption; 
known _— previous’ placenta; multiple gestation; 
preeclampsia/gestational hypertension; prior PPH; 
obesity; and anemia, while intrapartum risk factors 
include: retained placenta; operative vaginal birth, 
mediolateral episiotomy; prolonged labor; large infant 
(>4kg); and pyrexia during labor [5,17,20-22]. 


Being aware of antepartum and intrapartum risk 
factors is essential to preventing and managing 
potentially massive bleeding. The main cause of early PPH 
in 70% of cases is uterine atony, followed by uterine 
rupture, placenta retention, trauma of the soft birth canal 
(instrumental delivery, spontaneous trauma, episiotomy), 
disseminated intravascular coagulation, and uterine 
inversion, while the causes of late PPH _ include: 
endometritis and retention of ovary remains. The 
mnemonic of 4Ts is useful to remembering the causes: 
tonus, tissue, trauma, and thrombosis. 


PPH is the most significant complication during the 
puerperal pregnancy cycle, especially in the third stage of 
labor. It requires recognizing antepartum and 
intrapartum risk factors, as well as being aware of care 
actions able to prevent, manage, early diagnose, and treat 
the condition [21,23]. For that, the development and 
validation of an instrument intended to record timely and 
appropriate interventions will positively impact 
morbidity and mortality associated with this obstetrical 
complication. 


Thus, the objectives of this study included: to propose 
an instrument addressing the actions of nurses to prevent 
and manage hemorrhaging in the third stage of labor, and 
validate its content among nurse experts, by analyzing its 
psychometric properties so that preventive and control 
measures can be included in the routines of all workers 
assisting women during labor and deliveries. 
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Method 


This methodological, descriptive study with a 
quantitative approach was intended to validate the 
content of an instrument addressing nursing care 
technology in the prevention and management of 
hemorrhage in the third stage of labor. 


This type of study is intended to investigate methods 
to obtain, organize and analyze data, develop, validate and 
assess instruments and research techniques. Note that the 
goal of this type of study is to develop a reliable 
instrument that can be later used by other researchers 
[24-26]. 


The instrument’s content was validated in two stages: 
first, during the development of the instrument, while the 
second stage was the phase when experts quantified and 
assessed the process. An integrative and systematic 
review was conducted to identify the instrument’s 
content and establish how many domains and care actions 
would be necessary to organize such actions ultimately. 
The second stage consisted of content validation. The 
experts assessed the instrument and actions proposed 
according to the importance of the domains and their 
respective content [27]. In addition to the instrument’s 
content, clarity and objectivity, ten criteria proposed by 
Pasquali [28] were also assessed. 


The judges who validated content were identified from 
a list of Obstetric Nurses registered in the Brazilian 
Nursing Council as experts; a total of 195 professionals 
are listed. Inclusion criteria were: having a curriculum in 
the Lattes Platform with an email available; working in 
one of the following: an Obstetrical Unit; a Natural Birth 
Center; rooming-in; in a Planned delivery team; 
Undergraduate teaching; or high school teaching. 


Those who agreed to participate in the study were 
asked to confirm their consent through email and sign a 
free and informed consent form. If no response was 
obtained, a new invitation was resent every seven days 
until a sample of 20 participants was obtained. The 
recommendations provided by Pasquali [29] concerning 
the number of judges (from six to 20) were adopted. 


After agreeing to participate in the study, each expert 
completed an electronic form, developed via Google forms, 
divided into three parts: 

Part 1: Characterization of the judges. 

Part 2: Instrument to assess each item in regard to 
content. A total of 81 care actions were divided into 20 
domains based on the priority of nursing interventions, 


Teixeira Rangel, RC, et al. Content Validation of the Instrument Nursing 
Care Technology in the Prevention and Management of Hemorrhage in 
the Third Stage of Labor. Nurs Health Care Int J 2019, 3(3): 000190. 


Nursing & Healthcare International Journal 


which were assessed on a seven-point Likert scale. The 
experts rated how appropriate care actions proposed in 
each domain were (1=inappropriate and incomplete; 
2=requires major review to become appropriate and 
complete; 3=requires minor review to be appropriate and 
complete; 4=neither inappropriate, nor appropriate; 
5=appropriate and complete; 6=very appropriate and 
complete; 7=totally appropriate and complete). After this, 
the experts were invited to provide the content they 
considered necessary to be added to the domains. 


The form, developed online, was called: “Tecnologia do 
cuidado do enfermeiro na prevenc¢do e no controle da 
hemorragia no terceiro periodo do parto: validacdo de 
contetido" [Nursing Care Technology in the prevention 
and management of hemorrhage in the third stage of 
labor: content validation]. Different types of answers 
could be selected, for instance: checkboxes, drop-down 
lists, multiple choice, plain text, or paragraph text field, 
which allows respondents to provide their opinion and 
write longer answers. 


Part 3: Assessment of the instrument in terms of its 
composition. This part of the instrument addressed 10 
criteria recommended by Pasquali [28], namely: scope; 
clarity; coherence; criticality of items; objectivity; 
scientific redaction; relevance; sequence; homogeneity; 
and currentness. 


A seven-point Likert scale (ranging from 1-totally 
disagree to 7-totally agree) followed each of the 10 items. 
The answers provided by the experts were entered into a 
Microsoft Excel (2010) spreadsheet and the scores 
obtained by each item were computed. Cronbach’s alpha 
and the Content Validity Index (CVI) were used to validate 
the instrument in regard to the relevance of its items and 
the instrument as a whole, obtained by inter-rater 
agreement. Validation is obtained when a group reaches a 
consensus in regard to a given phenomenon. Such a group 
is composed of experts, that is, professionals effectively 
engaged in the field in which a study is being developed 
[30]. 


The Cronbach’s alpha index ranges from 0 to 1 and 
estimates how uniformly items contribute to the 
unweighted sum of the instrument. This property is 
known as internal consistency, hence, alpha is interpreted 
as the mean coefficient of all estimates of internal 
consistency one would obtain if performing all the scale’s 
possible divisions [31]. According to Freitas, et al. [32] 
and Freitas, et al. [33], alpha is considered satisfactory 
when a 2 0.70, which is the cutoff point used in this study. 
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Content validation is intended to measure whether the 
study represents a measure of reality such as it is. The 
theoretical analysis is performed by the experts and is 
intended to establish understanding of items and their 
pertinence to the construct one intends to measure [28]. 


According to Rubio et al. [34], the CVI assesses 
agreement among judges in regard to _ the 
representativeness of the measure in relation to the 
content it addresses; which is calculated dividing the 
number of judges who agreed with the item by the total 
number of judges (CVI of each item). 


Agreement or disagreement was verified; that is, the 
proportion of judges who rated the item as valid (with a 
score greater than 5 in the Likert scale) to remain in the 
protocol was verified in relation to the total number of 
judges. A CVI greater than 0.80 was necessary for an item 
to remain in the instrument [35]. 


Ethical guidelines were taken into account according 
to what is established in Resolution No. 446/2012, 
Brazilian Council of Health [36]. This study was submitted 
to and approved by the Institutional Review Board at 
UFSC, via Platform Brasil, No. 169.110 from October 10, 
2012 - CAAE n. 03586312.0.0000.0121. CONEP was also 
consulted (No. 120.343, de 08/10/2012). 


Results 


The 20 experts scored all the items using a seven-point 
Likert scale ranging from 1-totally disagree to 7-totally 
agree Table 1. 


Table 1 Judges’ assessments of the instrument nursing 
care technology in the prevention and management of 
hemorrhage in the third stage of labor in regard to its 
composition according to 10 criteria proposed by 
Pasquali [28]. Floriandpolis, SC, Brazil 2019. 


wn 2 |oz | os |e. | es SS Se | 2 = 
oS [-¥ _ wy — — = <x S S 
= S) a & S 5 g > 5 o 2 TOTAL % 
= a s = eS a = = o> = 4 
~ = Se | SB 2 | ee | es 
ss) ° = o 
1 6 5 5 5 5 5 7 6 7 7 58 83 
2 7 il 7 7 7 7 i 7 7 7 70 100 
3 7 7 7 7 7 6 7 7 7 7 69 98.6 
4 7 7 7 7 7 7 7 7 7 7 70 100 
5 7 6 6 7 6 7 6 7 5 6 63 90 
6 7 7 4 6 7 5 7 7 6 5 61 87.1 
7 7 7 7 7 6 7 7 7 7 fd 69 98.6 
8 5 5 5 6 5 5 7 5 4 6 53 75.7 
9 7 7 7 7 7 7 7 7 7 7 70 100 
10 6 5 7 7 7 7 7 6 i is 66 94.3 
11 6 6 7 6 7 6 6 6 6 7 63 90 
12 5 3 5 5 3 5 5 3 3 6 43 61.4 
13 7 7 7 6 6 7 7 5 6 7 65 92.8 
14 5 4 5 5 4 5 5 5 5 5 48 68.6 
15 7 7 7 7 7 7 7 7 7 7 70 100 
16 5 5 5 5 5 5 5 5 5 5 50 71.4 
17 id 5 7 7 7 6 7 7 7 id 67 95.7 
18 4 4 5 4 4 4 4 4 5 4 42 60 
19 7 6 7 7 7 7 7 7 7 7 69 98.6 
20 6 6 6 6 6 6 7 5 6 7 61 87.1 
TOTAL 125 | # 123 | 124 | 120 | 121 [| 129 | 120 [ 121 [| 128 | 4.WW _ 
% 89 | 83 | 879 | 89 | 85.7 | 864 | 921 | 85.7 | 86 | 914 


Source: Developed by the author (2019). 


Table 1: Presents the assessment of the experts in regard to the instrument’s composition according the 10 criteria 


proposed by Pasquali [28]. 
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The items relevance and currentness were the ones 
assessed the highest by the judges, with 92.1% and 91.4%, 
respectively; followed by the items scope, criticality and 
coherence, with approval rates of 89.3%, 88.6% and 
87.9%, respectively; while the items concerning scientific 
redaction and homogeneity obtained 86.4% approval; 
objectivity and sequence obtained 85.7%; and clarity was 


the item with the lowest level of approval, with 82.9%. 


Cronbach’s alpha and Content Validity Index (CVI) 
were the reliability measures used to assess the content of 
the instrument nursing care technology in the prevention 
and management of hemorrhage in the third stage of 
labor, conducted by 20 judges (Table 2). 
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Coherenceste! 6.2 1.01 0.95 0.96 - 

Criticality of items 6.2 0.93 0.95 0.95 9 

Objectivity 6 122 0.85 0.95 = 

Scientific redaction 6.1 0.97 0.95 0.96 s 
Relevance 6.5 0.92 0.95 0.96 
Sequence 6 1.18 0.9 0.96 
Homogeneity 6.1 1.16 0.9 0.96 
Currentness 6.4 0.92 0.95 0.96 

Total 6.1 1.05 0.9 1 Very High 


Table 2: Content validation using the reliability measures Cronbach’s alpha and Content Validity Index (CVI) of the 
instrument nursing care technology in the prevention and management of hemorrhage in the third stage of labor. 


Florianépolis, SC, Brazil 2019.N 20. 
Source: Developed by the author (2019). 


Table 2 presents the classification of items scope, 
coherence, criticality of items, scientific redaction, 
relevance and currentness, which were considered 
excellent, with 0.95; the items sequence and homogeneity 
were scored 0.90, while 0.85 was the score obtained by 
clarity and objectivity. 


The items with the highest classification, with a 
Cronbach’s alpha of 0.96, in terms of the instrument’s 
reliability were clarity, coherence, scientific redaction, 
relevance, sequence, homogeneity, and currentness. The 
items scope, criticality of items, and objectivity obtained a 
Cronbach’s alpha of 0.95; that is, the judges considered all 
items to have very high reliability. 
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The instrument nursing care technology in the 
prevention and management of hemorrhage in the third 
stage of labor presented a CVI equal to 0.92, which is 
considered excellent, and a Cronbach’s alpha of 0.96; the 
judges classified the axes, domains and actions as highly 
reliable. 


After assessing and analyzing data, the judges 
suggested new items addressing actions to be added to 
the domains: 2-Perform Physical/Obstetrical Assessment; 
4 - Monitor pregnant women after determining risk; 5- 
Manage labor with a partogram; 6-Identify and monitor 
failure to progress in labor; 7-Monitor and assess the use 
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of oxytocin; 10-Prevent perineal trauma in the second 
stage of labor; 14 - Monitor placental 
detachment/discharge in vaginal deliveries; and 17 - 
Communicate the multidisciplinary team in the event of 
hemorrhagic complications (placental retention or 
mother’s collapse). 


The judges’ suggestions to change 11 care actions 
were accepted to improve the instrument. For instance, 
the word patient was replaced by woman, pregnant 
woman, or parturient; that is, the nursing care instrument 
in the prevention and management of hemorrhage in the 
third stage of labor was adjusted according to the judges’ 
suggestions and based on evidence reported in the 
literature. Content was adjusted before completing and 
validating the final version. 


Discussion 


At the end of the content validation process that 
included the assessment performed by the experts, the 
instrument encompassed five axes, 20 domains and 92 
care actions to record the needs of pregnant women, in 
order to support care delivery. No item was excluded 
from the instrument after the analysis performed by the 
experts. The 10 criteria recommended by Pasquali [28] 
obtained Cronbach’s alphas (a) and Content Validity 
Indexes (CVI) well above the cutoff point established in 
the method adopted in this study, that is, Cronbach’s 
alpha = 0.70 and CVI = 0.80. Yamada, et al. [37] adopted a 
Cronbach’s alpha less than 0.70 to justify the exclusion of 
items of an instrument. 


Data were analyzed using statistical tests. Cronbach’s 
alpha was chosen because of its ability to reflect the level 
of agreement among items; the closer to one, the closer to 
100% for the correspondence of items. 


Even after validation, the assessment of criteria 
proposed by Pasquali [28,29] obtained values ranging 
from minimum to maximum. The items that obtained the 
highest Cronbach’s alpha, 0.96, were: clarity, coherence, 
scientific redaction, relevance, sequence, homogeneity, 
and currentness. Scope, criticality and objectivity 
obtained Cronbach’s alpha equal to 0.95. The judges 
considered that all the items presented very high 
reliability, which corroborates the instrument’s reliability 
and internal consistency, the estimated reliability among 
judges and content validation. The instrument as a whole 
obtained a Cronbach’s alpha equal to 0.96, while the 
judges considered the reliability of its axes, domains and 
actions highly reliable. 
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In a study validating a questionnaire addressing 
knowledge of asthma during childhood, Dourado, Maia 
and Aratjo [38] report that a Cronbach’s alpha equal to 
0.88, which was obtained for data concerning the 
caregivers’ assessment, was a_ Satisfactory index, 
considered by various studies to be an indication that 
data are relevant and pertinent. In this study, an even 
higher Cronbach’s alpha, 0.96, was found. 


In a study validating the content of a profile of nursing 
competencies of managers with higher education, 
Nogueira and Cunha [39] found Cronbach’s alphas 
between 0.79 and 0.88, that is, results were even lower 
than those obtained in this study. In a study intended to 
validate an educational game addressing sexuality among 
adolescents, Sousa, Oliveira and Coelho [40] found a 
Cronbach's alpha for the game as a whole equal to 0.88, 
again, a result that is lower than the one found here. The 
study by Silva, et al. [41] reports a Cronbach’s alpha equal 
to 0.76 for the content validation of a scale intended to 
predict hypertension complications. 


Cronbach’s alpha, chosen to assess this instrument’s 
reliability, enabled to verify internal consistency and the 
estimate reliability among the experts. The Cronbach’s 
alpha is based on the variance of individual items and 
covariance among items [32,33,42,43]. 


The instrument nursing care technology in the 
prevention and management of hemorrhage in the third 
stage of labor obtained a CVI equal to 0.95 in the items 
scope, coherence, criticality of items, scientific redaction, 
relevance and currentness; 0.90 in the items sequence, 
homogeneity; and 0.85 in clarity and objectivity. In 
general, suggestions in the literature are for a minimum 
agreement of 0.80 when validating new instruments [35]. 
The total CVI obtained in this study was 0.92. The general 
CVI obtained in the study conducted by Dourado, Maia 
and Araujo [38], which indicated their questionnaire was 
considered good to be applied to its target public, was 
0.76, a result lower than the one obtained in this study. 


The study concerning the content validation of the 
profile of nursing competencies of managers with higher 
education conducted by Nogueira and Cunha [39] 
obtained CVIs between 0.88 and 0.95. 


The study conducted by Sousa, Oliveira and Coelho [40] 
to validate an educational game addressing sexuality 
among adolescents obtained a CVI of 0.94. A CVI equal to 
0.98 was obtained by Silva et al. [41], when validating a 
scale intended to predict hypertension complications. The 
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results obtained in both these studies were higher than 
the one obtained in this study, which was 0.92. 


CVI assesses agreement among judges in regard to the 
representativeness of a measure in relation to its content, 
by dividing the number of judges who rated an item as 
relevant or extremely relevant by the total number of 
judges (CVI for each item), which results in the proportion 
of judges who rated an item to be valid [34]. 


Composition and criteria were analyzed considering 
the instrument’s reliability measure Cronbach’s alpha 
(total of 0.96) and Content Validity (total equal to 0.92), 
assessed by 20 obstetrical nurses who considered the 
instrument’s reliability as well as each of its items to be 
‘very high’; that is, the instrument was considered 
appropriate for use during care provided to pregnant 
women during labor, delivery and puerperium. 


Melo, et al. [44] highlighted that expert-nurses with 
experience in teaching, research, care delivery and 
management are essential for the validation of an 
instrument. Hence, recruiting more experienced 
professionals ensures greater accuracy in the assessment 
of instruments. 


Among future prospects, we highlight the validation of 
this instrument in other cultural contexts, as well as its 
clinical validation. Considering the importance of this 
technology, we will also register it as intellectual property. 


Conclusion 


This study enables understanding the process of 
content validation, essential for providing reliable 
instruments in the nursing field that support knowledge 
for safer clinical practice. Therefore, the validation of 
instruments with care actions is useful to properly 
perform appropriate nursing procedures. 


Statistical measures such as Cronbach’s alpha and 
Content Validated Index obtained very high values: total 
Cronbach’s alpha of 0.96 and total CVI of 0.92. Thus, the 
instrument validated in this study concerning nursing 
care technology used to prevent and manage hemorrhage 
in the third stage of labor is highly reliable. 
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